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(54) Image processing apparatus and method 

(57) An encoding error by the encoding and the de- 
coding of an image is controlled within a predetermined 
range by an image processing apparatus which includes 
an input unit for inputting image data, a prediction unit 
for predicting pixel values indicated by the image data, 
a generation unit for generating pixel values derived by 
converting the pixel values of the image data within a 
range of ±e based on the prediction result by said pre- 
diction unit and an encoding unit for encoding parame- 
ters relating to the pixel values generated by said gen- 
eration unit based on the prediction result by said pre- 
diction unit. 

An image having a gradient in the values of block 
pixels is encoded at a high efficiency by an image 
processing apparatus which includes a division unit for 
dividing an image into block pixels to be encoded, a gen- 
eration unit for generating parameters defining a plane 
based on values of first block pixels and values indicat- 
ing positions of the block pixels to be encoded and a 
plane encoding unit for encoding difference values be- 
tween the values of the block pixels indicated by the 
plane defined by the parameters and the values of the 
encoded block pixels. 
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Description 

The present invention relates to image processing 
apparatus and method for encoding an image. 

An image encoding method is classified into two 
major classes, an information preservation type encod- 
ing method in which an original image is completely re- 
stored when decoded and an information-non-reserva- 
tion type encoding in which an error from the original 
image is included. 

As the information preservation type encoding 
method, a DPCM encoding method has been known. In 
a common DPCM encoding, a predicted value of a pixel 
to be encoded is calculated based on encoded pixels 
around the pixel to be encoded, that is, a left pixel (A), 
an upper pixel (B) and a lift diagonal pixel (C) by using 
a formula (A+B-C), for example, and a difference be- 
tween the result and the value of the pixel to be encoded 
is encoded. 

In the information non-reservation type encoding 
method, it is not possible to completely restore the orig- 
inal image but it allows a high compression ratio. There- 
fore, when a large amount of image data is handled or 
an error is not critical, the information non-reservation 
type encoding system is frequently used. A JPEG base- 
line system is one of the information non-reservation 
type encoding methods. 

In prior art image compression, a block code has 
been frequently used. In a conversion encoding which 
includes many irrevocable encoding, a block is a unit of 
encoding. 

The concept of predicting a pixel value by a plane 
has been adopted in a commonly used DPCM. For ex- 
ample, for a pixel under consideration, a predicted value 
is calculated by (a+b-c) where a is a left adjacent pixel 
value, b is an upper pixel value and c is a left upper pixel 
value. 

In general, since most portions of the image are 
formed by simple brilliance gradients, the effect of using 
the above prediction is great. 

In the prior art prediction encoding system, the val- 
ue of pixel to be encoded is predicted from values of 
surrounding pixels and a difference between the predict- 
ed value and the value of the pixel under consideration 
is entropy -coded and transmitted. In this system, the 
amount of codes generated is influenced by the predic- 
tion system and an optimum prediction system differed 
from area to area. Accordingly, in order to efficiently ap- 
ply this system, it is common to use a method in which 
the image is divided into blocks and an optimal predic- 
tion system is selected for each block and a code is gen- 
erated by adding information for specifying the predic- 
tion system. 

The method for selecting the prediction system for 
each block is effective when the optimal prediction 
method differs from block to block but the transmission 
of the added information is not effective when the pre- 
dict ion system is uniform in a large area such as an im- 
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age background area. 

In the prior art image encoding, since the amount 
of encoded data is simply controlled, it is not possible to 
encode an image in which a certain degree of error be- 

s tween an actual pixel value of the image to be encoded 
and a pixel value of the encoded and decoded version 
of the original image is permitted (an allowable error 
range), at a high efficiency and within a predetermined 
error of the pixel value. 

10 A digital image which is actually handled is obtained 
by reading a printed document or a silver halide photo- 
graph by an image scanner or a CCD sensor, and in 
many cases, random noises are included over an entire 
area of the image. From experience, in an 8-bit tone bril- 

is liance data, random noise having a maximum amplitude 
of 3 is included. When the prediction is made based on 
the prior art method of a+b-c, a noise which has a larger 
dispersion than a dispersion value of the noise is includ- 
ed in the predicted value and hence the efficiency of the 

20 encoding is not enhanced. 

It is an aim of one aspect of the present invention 
to encode image data at a high compression ratio while 
controlling an error of a pixel value. 

In accordance with a preferred embodiment of the 

25 present invention, there is provided an image process- 
ing apparatus comprising input means for inputting im- 
age data, prediction means for predicting pixel values 
indicated by the image data; generation means for gen- 
erating pixel values derived by converting the pixel val- 

30 ues of the image data within a range of ±e based on the 
prediction result by the prediction means and encoding 
means for encoding parameters relating to the pixel val- 
ues generated by the generation means based on the 
prediction result by the prediction means. It is an aim of 

35 another aspect of the present invention to encode multi- 
value data at a high compression ratio while controlling 
an error of a pixel value. 

It is an aim of a further aspect of the present inven- 
tion to provide an encoding method which eliminates the 

40 problems in the prior art and allows encoding of data 
block by block, particularly a high efficiency reversible 
encoding of gradient image data, accordance with a pre- 
ferred embodiment of the present invention, there is pro- 
vided an image processing apparatus comprising divi- 

45 sion means for dividing an image into block pixels to be 
encoded, generation means for generating parameters 
defining a plane based on values of first block pixels and 
values indicating positions of the block pixels to be en- 
coded and plane encoding means for encoding differ- 
so ence values between the values of the block pixels in- 
dicated by the plane defined by the parameters and the 
values of the encoded block pixels. 

Embodiments of the present invention will now be 
described with reference to the accompanying draw- 

S5 ings, in which: 

Fig. 1 shows a flow of a process in a first form of 
implementation of one embodiment of the present 
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invention; 

Figs. 2A and 2B show types of an encoding method 

and a positional relation of pixels; 

Fig. 3 shows a portion of image data in the encoded 

data; 

Fig. 4 illustrates a Huffman table to encode cO'; 
Fig. 5 shows a method for scanning run length in a 
#8 encoding method; 

Fig. 6 show a manner of dividing a block to be proc- 
essed into sub-blocks; 
Fig. 7 shows a flow of an S3 process; 
Fig. 8 which is composed of Figs. 8A to 8C shows 
a flow of S21 , S25 and S32 processes; 
Fig. 9 shows a maximum block division method and 
encoded data format; 

Fig. 10 shows a manner of converting a difference 
value to a corrected difference value; 
Fig. 11 shows a flow of an S10 encoding process; 
Fig. 12 shows a circuit configuration to implement 
one embodiment of the present invention; 
Fig. 1 3 shows a circuit configuration to implement 
the first form of implementation; 
Fig. 14 shows a block diagram of an encoding ap- 
paratus of a first embodiment of the present inven- 
tion; 

Fig. 15 shows position of surrounding pixels a, b 

and c for a pixel under consideration; 

Fig. 16 illustrates block generation; and 

Fig. 17 shows a block diagram of a plane encoding 

circuit. 

[First Embodiment] 

Fig. 1 shows a flow of a process of a first embodi- 
ment of the present invention. In Fig. 1, SO denotes a 
step to set a reference Huffman table to be described 
later, S1 denotes a step to read image data, S2 denotes 
a step to read block data from the image data at a max- 
imum block size (16X16 pixels in the present embodi- 
ment), S3 denotes a step to determine a block division 
method and an encoding method, S4 denotes a step to 
store the block division method and the encoding meth- 
od determined in the step S3 as reference information, 
S5 denotes a step to calculate statistics of frequency of 
occurrence of difference value (prediction error) to be 
described later which is outputted based on the refer- 
ence information, S6 denotes a step to update the block 
data read in the step S2 to virtual block data to be de- 
scribed later, S7 denotes a step to determine an end 
point of an image, S8 denotes a step to generate an op- 
timal Huffman table based on the difference value sta- 
tistics calculated in the step S5, S9 denotes a step to 
determine a loop end and S1 0 denotes a step to encode 
the entire image based on the reference information. 

The present embodiment is now briefly explained. 
The encoding in the present embodiment is not a non- 
reversible encoding but an encoding method in which 
an error between a pixel value of an actual image and 
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a pixel value of an encoded image is admitted within a 
range of ±e at maximum. This is referred to as a near 
lossless encoding. 

It is assumed here that a maximum size of block to 
be encoded is 16X16 pixels and the encoding is per- 
formed at one of 16X16 pixels, 8x8 pixels and 4X4 pix- 
els. For the encoding method for each block, six types 
of prediction encoding methods (#0 to #5), two types of 
plane encoding methods (#6 and #7) and a run length 
encoding method (#8) which may be used only when 
the block is represented in binary form may be selected 
(see Figs. 2A and 2B). 

The nine types of encoding methods used for the 
encoding the block are briefly explained. Fig. 3 shows 
formats of code data by the respective encoding meth- 
ods. 

The #0 to #5 are for the prediction encoding method 
in which the predicted value is calculated using sur- 
rounding pixels A, B and C shown in Fig. 2A by the for- 
mulas of A+B-C, B, A, B+(A-C)/2, A+(B-C)/2, (A+B+C) 
/3, and a difference between the pixel under considera- 
tion and the predicted value is Huffman-encoded. The 
Huffman table is prepared for each block size and it is 
encoded by using a separate Huffman table for each 
block size. 

The #6 and #7 are for the plane encoding method. 
In the #6, a plane which approximates a pixel value of 
the block is determined by P(x, y)=ax+by+c0, where cO 
is a left upper corner pixel value in the block, and the 
values of a, b and cO and a difference between the ap- 
proximate plane and the pixel value in the block are Huff- 
man-encoded. 

If a and b are below a predetermined level, a differ- 
ence cO 1 from the previously appeared cO is Huffman- 
encoded forcO, and if they are above the predetermined 
level, the cO value itself is encoded. A flag is inserted 
after a, b and cO so that the difference data is not added 
when all of the reference values in the block are zero. 

The #7 is used when all pixel values in the block are 
equal, that is, when it is a plane of a=b=c, and the cO' 
described above is Huffman -encoded. Fig. 4 shows a 
Huffman table which is used when the cO' is encoded in 
the #6 and #7. In Fig. 4, S denotes a sign bit which is 0 
or 1 depending on the sign of the difference value cO. 
When the cO is no smaller than 1 4 or no larger than -1 4, 
the difference value is not used and the value of cO is 
added after the ESC code. 

The #8 is for the run length encoding method and it 
is used only when there are only two pixel values in the 
block. For example, assuming that a higher brilliance 
pixel represent white and a lower brilliance pixel repre- 
sents black, the pixels in the block are scanned in the 
order shown in Fig. 5, converted to a run length code 
and it is encoded. For the encoding, predetermined 
white (high brilliance) run and black (low brilliance) run 
Huffman table are used and the last run in the block is 
substituted by the EOB code. 

Referring to Fig. 1 , a flow of process in the present 
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embodiment is now explained in detail. 

In the step SO, the reference Huffman table to be 
used in encoding the block is set. A Huffman table pre- 
pared based on statistical information of predicted er- 
rors generated by prediction encoding determined for a 
plurality of images or by other method is read and a 
16X16 pixel prediction coding Huffman table, a 8x8 pix- 
el prediction encoding Huffman table, a 4X4 pixel pre- 
diction encoding Huffman table and a plane encoding 
Huffman table are set. 

Four counters to be used when those Huffman ta- 
bles are newly prepared in the step S8, that is, a 1 6x 1 6 
prediction encoding counter, a 8x8 pixel prediction en- 
coding counter, a 4X4 pixel prediction encoding counter 
and a plane encoding counter are all initialized to zero. 

A Huffman table for Huffman-encoding the informa- 
tion indicating the encoding method (#1 to #8) is pre- 
pared for each block size, an encoding counter to be 
used in newly preparing the Huffman tables is provided 
and it is also initialized to zero. 

In the step S1 , the image data to be encoded is read 
onto an encoding memory 

In the step S2, the block data (maximum block size 
16x16 pixels) are sequentially read from the image to 
be encoded. The upper and left lines of the read block 
are also read as reference areas for the prediction en- 
coding. 

In the step S3, for the maximum size block (16X16 
pixels) read in the step S1 , the block division method to 
further dividing the block to sub-block (blocks to be en- 
coded) appropriate for the encoding and the encoding 
method (#0 to #8) appropriate for the each block to be 
encoded are determined and the pixel values in the 
maximum block are updated within an error of ±e from 
the actual pixel values. In this manner, the encoding can 
be performed with a small difference value (prediction 
error). By updating the pixel value within the range of ±e 
so that the difference values have only several values, 
the amount of information of the difference values can 
be reduced. The process of the step S3 is described be- 
low in detail. 

The routine of the step S3 has a reversible structure 
and a total amount of codes when the maximum block 
or a cubic block derived by dividing the maximum block 
several times (which is refereed to as the block to be 
processed and the size thereof differs depending on the 
flow of process) is sub-divided (into four blocks) is com- 
pared with an amount of codes when such division is 
not made to determine whether the block is to be divided 
or not (see Fig. 6), and the reference information includ- 
ing the block division information and the encoding 
method as well as the amount of codes when the block 
selected by the reference information is encoded are 
outputted. 

Fig. 7 shows a detailed flow of the iterative process 
used when the encoding method of the maximum block 
(the block to be processed) and the division method are 
selected in the step S3. 
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In Fig. 7, S30 denotes a step to determine the size 
of the block to be processed. If the block to be processed 
is the minimum block size (4X4 pixels), the process pro- 
ceeds to S33, other wise the process proceeds to S20 

s and S23. 

S20 denotes a step to copy the block to be proc- 
essed to generate a block A. The block to be processed 
is copied to the block A in the memory area for the iter- 
ative process. S21 denotes a step to select the optimal 

w encoding method to minimize the amount of codes LA 
when the block A is encoded. Detail of the selection 
method will be described later. 

522 denotes a step to output the amount of codes 
LA generated when the encoding method selected in 

'5 S21 is used. 

523 denotes a step to copy the block to be proc- 
essed to generate a block B. S24 denotes a step to di- 
vide the block B into four sub-blocks and S25 denotes 
a step to select an optimal encoding method for each 
sub-block as is done in the step S21 . ~ 

S26 denotes a step to output the total amount of 
codes which is a sum of amounts of codes when the four 
sub-blocks are encoded by using the encoding methods 
selected in the step S25. 

In S27, the LA outputted in the step S22 is compare 
with the LB outputted in the step S29 and if LA<LB, the 
process proceeds to S28, and if LA^LB, the process pro- 
ceeds to S29. 

528 denotes a step to determine the size of the 
block to be processed in the block A as the size of the 
block to be processed, and the iterative process to de- 
termine the block division method for the range of the 
block to be processed is terminated. In S28 of the iter- 
ative process, the block A, that is, the block to be en- 
coded is updated to the pixel values (within the error 
range of ±e) of a virtual block to be described later gen- 
erated in selecting the encoding method in the step S21 . 
The process proceeds to S31 to generate the block di- 
vision information and the encoding method selected in 
the step S21 and the amount of codes LA as the refer- 
ence information. 

529 denotes a step to copy the block B comprising 
four sub-blocks to the range of the block to be processed 
used at the beginning of the iterative process. The four 
sub-bkxks are substituted by the blocks to be proc- 
essed and the iterative process to determine the block 
division method is iterated from the beginning. 

In S32, an optimal encoding method of the block to 
be processed having the minimum block size is select- 
ed. In S33, the amount of codes L is outputted. In S34, 
the block to be processed is updated by the pixel values 
of the virtual block to be generated later which is gener- 
ated in the step S32. In S35, the optimal encoding meth- 
od and the amount of codes L for the block are gener- 
ated. 

So far, the flow of the iterative process used when 
the encoding method of the maximum block (or the block 
to be processed) and the block division method are se- 
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lected in the step S3 has been described. 

The selection of the optimal encoding method in the 
steps S21 , S25 and S32 is now explained in detail. 

Figs. 8 A to 8C show a flow to select the optimal en- 
coding method. The amounts of codes of all selectable 
encoding methods are estimated to conduct the selec- 
tion of the optimal encoding method. 

Prior to the estimation of the amounts of codes, the 
block data is updated to a form appropriate for each 
method. In Fig. 8A, S40 denotes a step to initialize the 
amount of codes Ln(0) generated by the encoding meth- 
od #0, S41 denotes a step to determine a predicted val- 
ue A+B-C, S42 denotes a step to update the pixel values 
in the block within the error range of ±e, S43 denotes a 
step to add one pixel of Huffman code length for the dif- 
ference value to Ln(0), and S44 denotes a step to de- 
termine an end point of the block. When the amount of 
codes in encoding the block is generated, the process 
proceeds to S61 . 

The above precess is referred to mode 0 and mode 
1 to mode 5 correspond to the processes for the encod- 
ing methods #1 to #5 which are identical to the above 
process. 

S45 denotes a step to initialize the encoding meth- 
ods #6 and #7. A flag (FLAG) indicating whether the ap- 
proximate plane and the pixel values in the block com- 
pletely match or not is prepared and it is initially set to 
1 (complete match). The parameter Ln(6) indicating a 
code length for #6 is set to 0 and a parameter Ln(7) in- 
dicating a code length for #7 is set to a value which is 
not minimum of the Ln(0) to Ln(8). S46 denotes a step 
to determine whether the entire block is rendered to the 
same pixel value or not by changing the pixel values in 
the block within the error range ±e, S47 denotes a step 
to determine the block approximate plane P(x, y) = ax 
+ by + cO of the #6 described above, S48 denotes a step 
for each pixel to update the pixel value within the error 
range ±e to generate a pixel value for forming the virtual 
block and S49 denotes a step to add the Huffman code 
length for the difference value between the generated 
pixel value and the approximate plane P(x, y) to the Ln 
(6). S50 denotes a step to determine whether the differ- 
ence value (prediction error) is 0 or not. When there is 
even one pixel value which is different from that of the 
approximate plane, the FLAG is set to 0 in S51 . In S52, 
the end point of the block is determined. 

S53 denotes a step to determine the value of the 
FLAG. If FLAG = 0, the amount of codes of the difference 
value for each pixel generated in S49 and the amount 
of codes parameters a, b and cO (cO 1 ) indicating the ap- 
proximate plane P(x, y) are added to produce Ln(6) 
(S54), and the process proceeds to S61 . 

If FLAG = 1 (the approximate plane and the pixel 
values of the block completely match), only the informa- 
tion of the approximate plane need be encoded and the 
amount of codes of the parameters a, b, cO (cO') are sub- 
stituted by Ln(6) (S55), and the process proceeds to 
S61. 



When the value of Ln(6) is determined, Ln(7) is set 
to a value which is not minimum of Ln(0) to Ln(8). 

In S56, the amount of codes of cO (cO') is set to the 
value of Ln(7) and Ln(6) is set to a value which is not 
5 minimum of Ln(0) to Ln(8). In S64, the block data is up- 
dated (to generate the virtual block) within the range of 
±e such that all pixel values in the block are rendered 
to cO and the process proceeds to S61 . 

S57 denotes a step to determine whether the block 
io is represented by two pixel values or not. If it is repre- 
sented by the two pixel values, the process proceeds to 
S58, otherwise Ln(8) is set to a value which is not min- 
imum of Ln(0) to Ln(8), and the process proceeds to 
S61. 

15 S58 denotes a step to convert the pixel values to a 
run length. The amount of codes Ln(8) generated when 
the run length is encoded is determined and the process 
proceeds to S61. 

S61 denotes a step to compare the values of Ln(0) 

20 to Ln(8) to select a minimum value and the correspond- 
ing optimal encoding method. S62 denotes a step to 
store the virtual block corresponding to the encoding 
method selected in S61 of the virtual blocks generated 
by updating the pixel values within the range of ±e in the 

25 respective modes (mode 0 to mode 8), and S63 denotes 
a step to output the optimal encoding method and the 
amount of codes. 

The process before the step S61 determines the 
amounts of codes for the respective methods. This proc- 
ess is conducted after the block to be processed (or the 
sub-block) is copied in the separate memory area. Ac- 
cordingly, the updating of the block data when the opti- 
mal encoding method is selected does not affect to the 
original block to be processed. 

3S The steps S40 to S44 are now explained in detail. 
The steps S40 to S44 determine the amount of codes 
Ln(0) generated by the #0. 

S41 to S44 denote a loop to raster-process the pix- 
els in the block sequentially from the left upper and the 

40 processed pixels (the pixels updated in S42) are substi- 
tuted by the original pixel value as the reference pixels 
of the pixels under consideration. When the reference 
pixels, (A, B, C) are out of the area of the 16x16 pixel 
block read in S2, the pixel values of the block data up- 

45 dated in S6 before the loop precess (S2 to S7) of the 
previous maximum block is used. 

In the step S40, Ln(0) is set to 0. In the step S41, 
the predicted value P=A+B-C is generated based on the 
pixel value X under consideration and the surrounding 

50 pixel values A, B and C and the difference value D=X- 
P is calculated. In the step S42, a corrected difference 
value D' is determined by a formula (1) and the pixel 
under consideration is updated to D'+P. In the formula 
(1 ), ixi represents omission of digits after the decimal 

55 point. 



D*=i(D+a)/(2e+1 )ix (2e+1 ) 
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Fig. 1 1 illustrates a corrected difference value D' 
when e=3 and the difference value D=4. As shown in 
Fig. 1 1 , since the number of the difference values D'(-7, 
0, 7) is smaller than the number of difference values D 
(-10 to 10), the information indicating the difference val- 
ues is significantly reduced. Further, the possibility of 
the complete matching of the difference values and the 
predicted values is high. 

In the present embodiment, the pixel value to be 
corrected (the corrected pixel value) X*=X+3. In the step 
S43, the difference between the updated corrected pixel 
value X* and the predicted value R that is, the corrected 
difference value D' is Huffman-coded by the Huffman 
table for the size of the block to be processed (or the 
sub-block) and the generated code length is added to 
Ln(0). In the step S44, whether the process has been 
completed to the end point of the block or not is deter- 
mined, and if it is completed, the process proceeds to a 
step S61 , and if it is not completed, the next pixel is con- 
sidered and the process starting from the step S41 is 
repeated. In this manner, the amount of codes Ln(0) 
generated when the encoding method #0 is used is de- 
termined. For the encoding methods #1 to #5, only the 
prediction methods are different and the methods for de- 
termining the amount of generated codes Ln(l to 5) are 
same as that for the #0 and the detailed description 
thereof is omitted. 

The steps S45 to S56 and S64 are now explained 
in detail. S45 denotes a step to initialize the variables 
for the #6 and #7. The amount of codes Ln(6) generated 
by the #6 is set to 0 and the amount of codes Ln(7) gen- 
erated by the #7 is set to a sufficiently large value. A 
variable FLAG which is used to determine whether all 
difference values are 0 or not in the #6 is set to 1 . In a 
step S46, a maximum value (max) and a minimum value 
(min) of the pixel values in the block are determined, 
and if max-min is smaller than 2e+1, the process 
branches to a process for the encoding method for the 
#7 (steps S56 et seq), otherwise the process proceeds 
to a process for the encoding method #6 (steps S47 et 
seq). 

In a step S47, an approximate plane is determined 
from the pixel values in the block. The steps S48 to S52 
sequentially process the pixel values in the block. In a 
step S48, the difference value D=X-P is determined from 
the pixel values X under consideration in the block and 
the value P corresponding to the position of the pixel 
under consideration on the approximate plane deter- 
mined in the step S47, the corrected difference value D' 
is calculated in the same manner as that in the step S42 
and the pixel under consideration is updated to D*+R In 
a step S49, the difference value between the updated 
corrected pixel value X' and the predicted value P is 
Huffman-encoded by the Huffman table for the plane en- 
coding and the code length thereof is added to Ln(6). 
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In a step S50, whether the difference value between 
the corrected pixel value X? and the predicted value P is 

0 or not, and if it is not 0, the FLAG is set to 0 in a step 
S51. In a step S52, the end point of the block is deter- 

s mined, and if it is not the end point, the process starting 
from the step S48 is continued for the next pixel in the 
block. S53 denotes a step to determine the value of the 
FLAG. If it is 0, the process proceeds to S54, otherwise 
the process proceeds to S55. In the step S54, the 
io amount of codes of a, b and cO plus one bit for the FLAG 
is added to the Ln(6). In the step S55, the amount of 
codes of a, b and cO plus one bit for the FLAG is substi- 
tuted by a new Ln(6). 

If it is determined that the encoding method #7 is 
is available in the step S46, a quotient of (max-min)/2 is 
set to cO in a step 556 and the amount of codes when 
cO is encoded is substituted for the Ln(7) and a suffi- 
ciently large value is set to the Ln(6). 

In a step S64, all pixel values in the block are up- 
20 dated. By this process, the amounts of codes Ln(6) and 
Ln(7) when the encoding methods #6 and #7 are used 
are determined. 

The steps S57 to S60 are now explained in detail. 
S57 denotes a step to determine whether the encoding 
method #8 is available or not. The pixel values in the 
block are examined and if two pixel values appear, the 
process proceeds to S58, otherwise the process pro- 
ceeds to S50. 

In S50, a sufficiently large value is set to the Ln(8). 
In a step S58, the block is scanned in the order shown 
in Fig. 5 while a higher brilliance value of the two pixel 
values of the block is set to white and a lower brilliance 
value is set to black to convert it to white-and-black run 
length data. In a step S59, the amount of codes to rep- 
resent the two appearing pixel values and the amount 
of codes when the run length data is Huffman -encoded 
by using the Huffman table exclusively prepared for the 
#8 are added to determine the amount of generated 
codes Ln(8) for the #8. 

In this manner, the amounts of generated codes Ln 
(0) to Ln(8) for the encoding methods #0 to #8 are de- 
termined, and in a step S61, a minimum amount of 
codes which is the minimum of the above is determined 
and a sum of the minimum amount of codes and amount 
of codes indicating the encoding method is stored as the 
amount of code I of the block (one of the four amounts 
of codes comprising the amount of code LA or the total 
amount of code LB in Fig. 7, or the amount of code L) 
and the information representing the encoding method 
number (i) is also stored. 

In a step S62, the virtual block generated in using 
the encoding method #i is stored. The virtual block is 
used to determine the block to be encoded (update the 
original image values) in the steps S28 and S34, and it 
is finally used to update the block data in the step S6. 
In a step S63, the number i and the amount of codes 

1 are outputted and the optimal encoding method selec- 
tion process (corresponding to S21 , S25 and S32 of Fig. 
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7) is terminated. The description of the selection of the 
optimal encoding method in the steps S21 , S25 and S32 
is thus completed. 

In this manner, the block division information and 
the optimal encoding method in the step S3 have been 
determined. 

Fig. 9 shows an example of the encoding method 
for each of the divided blocks to be encoded. The refer- 
ence information based on Fig. 5 is shown in Fig. 10. 
This completes the detailed description of the step S3. 

In a step S4, the reference information generated 
in S3 is stored in the memory. 

In a step S5, for the encoding method number i se- 
lected for each of the blocks to be encoded, the numbers 
of times of occurrence of the encoding methods #0 to 
#8 are counted by encoding number counters provided 
one for each block size, and if it is one of #0 to #6, the 
number of times of occurrence of the corrected differ- 
ence value D' when encoded by that method is added 
to the counter corresponding to the encoding method 
(the prediction encoding counter for #0 to #5, and the 
plane encoding counter for #6). 

In a step S6, the pixel values of the block read in 
the step S2 are updated by using the data of the block 
to be encoded which were updated in the virtual block 
in the step S3 (based on S28 and S34). 

In a step S7, the end point of the image is deter- 
mined, and if it is not the end point of the image, the 
process staring from the step S2 is conducted to the next 
block, and if it is the end point, the process proceeds to 
a step S8. 

The loop precess of S2 to S7 is the process to up- 
date the block data (virtual block) for each block to gen- 
erate the image data. When the reference pixels (A, B, 
C or cO) used in the selection process of the encoding 
method in the step S3 are out of the area of the 16X16 
pixel block read in the step S2, the pixel values of the 
block data updated in the step S6 before the previous 
loop precess are used. 

In a step S8, based on the values of the 1 6X1 6 pixel 
prediction encoding counter, the 8x8 pixel prediction 
encoding counter, the 4x4 pixel prediction encoding 
counter and the plane prediction counter, the 16X16 pix- 
el prediction encoding Huffman table, the 8X8 pixels 
prediction encoding Huffman table, the 4X4 pixel pre- 
diction encoding Huffman table and the plane encoding 
Huffman table are reconstructed. Further, the numbers 
of times of occurrence of the optimal encoding methods 
#0 to #8 are counted by the encoding method number 
counters provided for the respective block sizes, and the 
Huffman table for encoding the encoding method num- 
bers are reconstructed in accordance with the statistics 
of the counts. 

In a step S9 ( the end of the optimization processing 
loop (SO to S8) is determined, and when a predeter- 
mined number of loops is completed, the process pro- 
ceeds to the step S9, otherwise the process proceeds 
to the step SO. In this case, the SO in the initial optimi- 



zation processing loop and the step SO in the second 
and subsequent optimization steps are different. In the 
SO of the second and subsequent loops, the optimal 
Huffman table prepared in the previous step S8 is sub- 

5 stituted for the reference Huffman table. 

In a step S10, the image data updated in the step 
S6 is actually encoded by referring the stored reference 
information. Fig. 12 shows a flow of the encoding proc- 
ess in the step S10. In Fig. 12, S80 denotes a step to 

10 encode the Huffman table, S81 denotes a step to read 
block data of the maximum size (1 6x16 pixels) from the 
image, S82 denotes a step to read the reference data 
corresponding to the block data, S83 denotes a step to 
encode the block data in accordance with the reference 

is information and S84 denotes a step to determine the 
end point of the block. Detailed description follows. 

In the step S80, for the 16X16 pixel encoding 
number Huffman table, the 8X8 pixel encoding number 
Huffman table, the 4X4 encoding number Huffman ta- 

20 ble, the 1 6X 1 6 pixel prediction encoding Huffman table, 
the 8x8 prediction encoding Huffman table, the 4X4 
prediction encoding Huffman table and the plane encod- 
ing Huffman table, the information indicating the struc- 
tures of the respective Huffman tables is outputted as 

2S the Huffman table information. 

In the step S81 , the block data are sequentially read 
with the maximum block size from the image data. In the 
step S82, the reference information for the block data is 
read from the memory. 

30 In the step S83, the reference information and the 
dataderived by encoding the block data of the maximum 
block size in accordance with the reference information 
are outputted as the encoded data sequence. 

In the step S84, the end point of the image is deter- 

35 mined, and if an unprocessed block is present, the proc- 
ess starting from the step S81 is conducted for the next 
block. 

In this manner, the encoded data sequence for the 
image data is generated and outputted. 
40 This completes the description of the first embodi- 
ment. 

[Modification of First Embodiment] 

45 The present invention is not limited to the above em- 
bodiment. For example, while the block is divided with 
the maximum block size of 16X16 pixels in the above 
embodiment, the block size may be set to 128x128 or 
larger pixel. 

so Further, while the block is divided into four same 
size sub-blocks in the above embodiment, the block size 
may be variable. 

In the first embodiment, the block is divided based 
on the amount of codes of the block to be processed 

ss and the amounts of codes of the sub-blocks derived by 
dividing the block to determine the block to be encoded. 
Alternatively, the amount of codes of all blocks from the 
maximum block (16X16 pixels) to the minimum block 
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(4X4 block) may be determined and then the block may 
be divided based on the amount of all codes. 

Further, the amount of codes of the minimum block 
may be first determined, this block is regarded as the 
sub-block, a temporary block division method and the 
amount of codes are determined based on the total 
amount of codes of the four sub-blocks (4X4 pixels) as 
shown in Fig. 6 and the amount of codes of the block 
(8x8 pixels) of the combined area, a new temporary 
block division method is determined based on the total 
amount of codes of the four determined areas (8x8 pix- 
els) and the amount of codes of a larger block (16X16 
pixels) comprising the determined area (8x8 pixels), 
and it is repeated until the maximum block size is 
reached to determine the block to be encoded. 

Further, while the prediction encoding method, the 
plane encoding method and the run length encoding 
method are used as the selectable block encoding 
method in the above embodiment, the application meth- 
od may be changed by changing the prediction method 
or other encoding method such as the Marcof model en- 
coding method may be used. While the Huffman encod- 
ing is uses as the entropy encoding method in the above 
embodiment, other entropy encoding method such as 
arithmetic encoding method may be used. 

In S6, the block data need not be updated but the 
value of the prediction error (generated in S3) when the 
image by the updated block data is encoded may be 
stored. In this case, in S10, instead of reencoding by 
using the updated block data, the encoding may be 
made by using the prediction error (corrected difference 
value) determined in the step S3. However, when the 
difference values for the respective pixels are stored, 
the circuit configuration and the process to generate the 
reference pixel values are complex compared to the 
process in which the prediction encoding is conducted 
by referring the pixel values (A, B and C) in S41 of Fig. 
8A. Accordingly, in the first embodiment, the method of 
storing the corrected pixel value is used rather than the 
method for storing the corrected difference value. 

Fig. 1 3 shows an example of the apparatus for con- 
ducting the encoding process of the first embodiment. 
Numeral 1 denotes a CPU control unit which can con- 
duct the encoding and decoding of the image data by 
the software processing, numeral 2 denotes an image 
input unit for reading the image data, numeral 3 denotes 
a printer unit for printing the image data, numeral 5 de- 
notes a communication control unit for transmitting the 
data encoded by the CPU control unit 1 or the encoded 
data transmitted from an external equipment and nu- 
meral 6 denotes an image memory for storing the image 
information. 

[Second Embodiment] 

Fig. 1 4 shows a block diagram of a second embod- 
iment of the present invention. In Fig. 14 numeral 100 
denotes an image input unit for generating image data 



such as an image reader, a TV camera or a host com- 
puter, numeral 101 denotes a block generation circuit 
for blocking image data for each pixel, numeral 103 de- 
notes a plane encoding circuit, numeral 104 denotes a 

5 buffer for adjusting timing, numeral 1 05 denotes a sub- 
block generation circuit for dividing the block in the block 
generation circuit into sub-blocks, numerals 106, 107, 
108 and 109 denote prediction encoding circuit for con- 
ducting the prediction encoding in different methods, nu- 

io meral 110 denotes a selector, numeral 111 denotes an 
information addition circuit, numeral 112 denotes a buff- 
er for adjusting the timing, numeral 113 denotes a se- 
lector, numeral 114 denotes an information addition cir- 
cuit, numerals 1 1 5, 1 1 6, 1 1 7 and 118 denote signal lines 

* 5 and numeral 119 denotes an output unit for outputting 
the encoded data to an external memory and an external 
equipment. 

Fig. 15 shows positions of surrounding pixels a, b 
and c used by the prediction encoding circuits 106, 107, 
20 1 08 and 1 09. In Fig. 1 5, x represents a pixel under con- 
sideration and a, b and c are defined by the relative po- 
sitions to x. 

First, the image data to be encoded is stored in the 
memory 101 from the image input unit 100. The block 

2S generation circuit 102 sequentially reads the image data 
by 1 6x 1 6 pixel block and stores them. As shown in Fig. 
3, the data of the left and upper pixels Qf the 16X16 pixel 
block are also read as shown in Fig. 3 and they are 
stored. The plane encoding circuit 103 calculates a 

30 plane ccx+py+y (x, y=0 15) which proximates the 

16X16 pixel block stored in the block generation circuit 
1 02 and Huffman-encodes the values of the parameters 
a, p and y and the difference value between the proxi- 
mate plane and the pixels values in the block and stores 

35 them in the buffer 1 04. Detail of the plane encoding cir- 
cuit 103 will be described later. 

On the other hand, the sub-block generation circuit 
105 sequentially reads the 8X8 pixel blocks from the 
16X16 pixel block stored in the block generation circuit 

40 1 02 and stores them. The data of the left and upper pix- 
els of the 8x8 pixel block are also read and stored. The 
prediction encoding circuit 106 calculates x-a for each 
pixel in the 8x8 pixel sub-block stored in the sub-block 
generation circuit 105 and Huffman-encodes it. Similar- 

45 fy, the prediction encoding circuit 1 07 Huffman-encodes 
x-b, the prediction encoding circuit 108 Huffman-en- 
codes x-c and the prediction encoding circuit 109 Huff- 
man-encodes x-a-b+c. The selector 110 selects the en- 
coded data of the smallest amount of codes of the en- 

so coded data outputted from the prediction encoding cir- 
cuits 1 06, 1 07, 1 08 and 1 09, outputs the code sequence 
thereof to the signal line 116 and outputs 2-bit identifi- 
cation data I to the signal line 1 1 5 to identify the selected 
prediction encoding circuit. The information addition cir- 

ss cuit 1 1 1 adds the identification data I from the signal line 

1 15 to the head of the code sequence of the signal line 

116 and stores them in the buffer 112. This process is 
conducted for the four pixel sub-blocks so that one block 
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of codes of the 1 6x 1 6 pixels are stored in the buffer 1 1 2. 

The selector 11 3 compares the amount of codes of 
the buffer 104 and the buffer 112, selects the smaller 
one and outputs it to the signal line 118. It also outputs 
one-bit identification data II for identifying the selected 5 
buffer 1 04 or 1 1 2 to the signal line 117. The information 
addition circuit 114 adds the identification data II from 
the signal line 1 1 7 to the head of the code sequence and 
outputs them to the output unit 119. 

The above process is repeated until the last block 'o 
stored in the memory 101 is reached to generate the 
code. 

Fig. 1 7 shows a configuration of the plane encoding 
circuit 103. 

In Fig. 17, numeral 201 denotes a plane calculation 
circu it, numeral 202 denotes a plane data generation cir- 
cuit, numeral 203 denotes a buffer, numeral 204 de- 
notes a differentiation circuit, numeral 205 denotes a pa- 
rameter encoding circuit, numeral 206 denotes a Huff- 
man encoding circuit and numeral 207 denotes a multi- 20 
plexor. 

In the present embodiment, the image data to be 
encoded is an 8-bit/pixet tonality image. 

The encoding is conducted block by block. The 
1 6X1 6 pixel block data from the block generation circuit 25 
201 is sent to the plane calculation circuit 201 and the 
buffer 203 through the signal lines 208 and 209. 

The plane calculation circuit 201 calculates the pa- 
rameters a, b, c9 and m defining the plane in a manner 
to be described later and sends them to the plane gen- 30 
eration circuit 202 and the parameter encoding circuit 
205 through 210, 211 and 212. The plane generation 
circuit 202 generates the plane data in accordance with 
the received parameters and outputs it to 21 4. A differ- 
ence between it and the input block image data from 35 
21 3 is calculated for each pixel in the block by the dif- 
ferentiation circuit 204 and the plane prediction error val- 
ue is sent to the Huffman encoding circuit 206 for each 
pixel. The Huffman encoding circuit 206 allocates the 
Huffman code in accordance with the built-in Huffman *o 
table and outputs the code data to 21 7. 

On the other hand, the parameter inputted to the 
parameter encoding circuit 205 from the signal line 211 
is encoded by an appropriate number of bits and it is 
sent to the multiplexor 207 from 216. The multiplexor 4S 
207 rearrange the parameter code sent from 216 and 
the difference code sent from 21 7 in a predetermined 
sequence to generate a series of bit stream which is out- 
putted to the buffer 104. 

• An operation of the plane calculation circuit 201 is so 
now explained. It is assumed that the pixel value of the 
left upper pixel of the input NXN (N=16 in the present 
embodiment) pixels is c8. When the pixel value in the 
pixel coordinate (x, y) in the block is dxy, the following 
statistics is taken. 
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j 1 



d = EE 2 ic e-d tJ ) j 

j i 

By using those formulas, 

a=(BD-2AC)/(4A 2 -B 2 ) 

b=(BC-2AD)/(4A 2 -B 2 ) 

are calculated and an integer m=minf max(lal, Ibl) ,1 
16) is determined, where I I indicates an absolute value 
and r 1 indicates round-up, max( ) indicates a larger 
one and min( ) indicates a smaller one. Thus, m as- 
sumes 1 to 16. Further, a and b are normalized and 
quantized by using m. 

a= L(a/m+1)128J 



b=L (b/m+1)128j 

where L J indicates omission. 
Thus, a and b are quantized to values of 0 to 255. 

When the initially calculated m is 1 6, a and/or b can 
assume a value smaller than 0 or larger than 255 and 
hence it is clipped to keep the range of 0 to 255. After a 
and b are rendered to 8-bit values, 1 is subtracted from 
m so that m is rendered to a 4-bit value of 0 to 15. 

The result from the above operation is a parameter 
which defines a best fit plane based on a minimum 
square error when the plane passing through the left up- 
per pixel of the block is given by Z=ax+by+ce and the 
position of the left top pixel is (0, 0 f C8). 

When all parameters have been calculated, the 
plane calculator 201 outputs 8-bit values for a, b and CO 
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and a 4-bit value for m. 

An operation of the plane generation circuit 202 is 
now explained. 

The plane generation unit 202 receives the plane 
parameters a, b, C0 and m explained above from 106 
to generate the prediction block. Namely, a' and b f are 
calculated by 

a*=(m+1)(a-128)256 



b'=(m+1)(b-1 28)256 



and the pixel value djj of the coordinate (i, j) (where i and 
j are integers between 0 and N-1 ) is generated by 



dy=L (Ce+a* i+b' j+0.5)J 



The block data generated in this manner is outputted to 
214. 

While all pixels in the block are encoded in the 
above embodiment, (1) the block may be divided into 
sub-blocks and the code indicating whether it is the sub- 
block to be encoded or not may be added for each sub- 
block, or (2) a plane corresponding to one-bit block may 
be provided to determine whether it is the sub-block to 
be encoded or not. The method briefly described in this 
supplement is particularly effective to a relatively com- 
plex portion in the image. 

In accordance with the plane encoding described 
above, the prediction method which does least cause 
the reduction of the encoding efficiency is attained for 
the simple brilliance gradient portion bearing noise 
which occupies a large portion of the actual image, and 
the encoding efficiency to the entire image is significant- 
ly improved over the prior art. 

[Modification of Second Embodiment] 

The present invention is not limited to the above em- 
bodiment. For example, while four values a, b, c and 
a+b-c are recited as the prediction values used in the 
prediction encoding in the above embodiment, any other 
combination which is constructed from the referable pix- 
els may be used. While the Huffman encoding is used 
together with the plane encoding and the prediction en- 
coding in the above embodiment, other entropy encod- 
ing system such as the arithmetic encoding may be 
used. The plane encoding is not limited to the embodi- 
ment in which a, p and y of the proximate plane ox+Py+y 
are encoded as the plane defining information but any 
values which can specify the plane may be used. Alter- 
natively, instead of the plane, a relatively simple curved 
surface may be used. 

The image encoding apparatus of the second em- 
bodiment comprises block division means for dividing 



the image data into blocks, plane encoding means for 
determining the proximate plane for each divided block 
and encoding the information specifying the proximate 
plane and the difference value between the proximate 

5 plane and the block, means for sub-dividing the block 
into sub-blocks, prediction encoding means for select- 
ing the optimal prediction system for each sub-block and 
encoding the information specifying the selected predic- 
tion system and the error by the selected prediction sys- 

10 tern, means for selecting one of the plane encoding 
means and the prediction encoding means for each 
block and means for adding the information for specify- 
ing the selected encoding system. Accordingly, the 
plane encoding for the block having the planar nature 

is and the prediction encoding for each sub-block for other 
block may be selectively used. Thus, the amount of gen- 
erated codes can be suppressed. 

By making the block size of the plane encoding to 
be larger than the block size of the prediction encoding, 

20 the efficient encoding is attained by utilizing the charac- 
teristic of the plane encoding. 

By extracting the information specifying the proxi- 
mate plane as the coefficients of the operation formula, 
the efficiency of the encoding by block by block is en- 

25 hanced. 

By converting the pixel values in the image or the 
block, the image data can be encoded with a relatively 
high efficiency while controlling the error of the pixel val- 
ues by using the relatively simple circuit configuration 
30 and process. 

Further, the multi-value data can be encoded at the 
high efficiency while controlling the error of the pixel val- 
ues. 

Further, since the image data to be encoded is up- 
35 dated prior to the encoding within the range of ±e, the 
encoding unit can confirm the decoded image. In this 
case, if it is after S6, the updated image data may be 
read and monitored on the display to confirm the decod- 
ed image. 

40 The image is divided into blocks, the formula of the 
proximate plane which proximate the block is calculated 
for each block and the error from the proximate plane is 
encoded. Thus, when the original image free from the 
actual noise is sufficiently proximated by the plane, the 

45 dispersion of the noise bearing on the prediction value 
can be suppressed to the same degree as that of the 
noise originally included in the original image and the 
reduction of the encoding efficiency is prevented. 

The encoding by using the proximate plane for each 

50 block and the encoding by the prediction method select- 
ed from the plurality of prediction methods are provided 
so that the high efficiency encoding is attained even 
when the multi-value data is reversibly encoded. 

The present inventbn may be modified without de- 

55 parting from the scope of the claims. 
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Claims 

1. An image processing apparatus comprising: 

input means for inputting image data; 
prediction means for predicting pixel values in- 
dicated by the image data; 
generation means for generating pixel values 
derived by converting the pixel values of the im- 
age data within a range of ±e based on the pre- 
diction result by said prediction means; and 
encoding means for encoding parameters re- 
lating to the pixel values generated by said gen- 
eration means based on the prediction result by 
said prediction means. 

2. An image processing apparatus according to Claim 
1 , wherein the prediction by said prediction means 
conducts a plurality of prediction methods. 

3. An image processing apparatus according to Claim 
1 , wherein the parameters relating to the pixel val- 
ues comprise difference values between the pixel 
values generated by said generation means and the 
prediction values predicted by said prediction 
means. 

4. An image processing apparatus according to Claim 
1 , wherein the pixel values indicated by the image 
data are represented by multi-value. 

5. An image processing apparatus according to Claim 
1 , wherein the encoding by said encoding means is 
conducted block by block. 

6. An image processing method comprising the steps 
of: 

inputting image data; 

predicting pixel values indicated by the image 
data; 

generating pixel values derived by converting 
the pixel values of the image data within a range 
of ±e based on the prediction result of said pre- 
diction means; and 

encoding parameters relating to the pixel val- 
ues generated in said generation step based on 
the prediction result of said prediction means. 

7. An image processing apparatus comprising: 



20 

of the block pixels within a predetermined 
range; and 

encoding means for encoding the block pixels 
converted by said conversion means based on 
5 the parameters. 

8. An image processing apparatus according to Claim 
7, wherein the parameters represent a three-dimen- 
sional plane, two dimensions of the three dimen- 

io sions represent a position of the block pixel and one 
dimension represents a pixel value. 

9. An image processing method comprising the steps 
of: 

15 

dividing an image into a plurality of block pixels; 
generating parameters indicating prediction 
values of the block pixels; 
converting the values of the block pixels while 
20 controlling differences between the prediction 

values and the values of the block pixels within 
a predetermined range; and 
encoding the block pixels converted by said 
conversion step based on the parameters. 

25 

10. An image processing apparatus comprising: 

input means for inputting image data; 
prediction means for predicting pixel values in- 
30 dicated by the image data; 

encoding means for encoding difference values 
between the pixel values indicated by the im- 
age data and prediction values predicted by 
said prediction means; and 
35 control means for controlling the difference val- 

ues. 

1 1 . An image processing apparatus according to Claim 
10, wherein the prediction by said prediction means 

40 uses a plurality of prediction methods. 

12. An image processing apparatus according to Claim 
1 0, wherein the difference values are set to discrete 
integers. 

45 

13. An image processing apparatus according to Claim 
10, wherein the pixel values indicated by the image 
data are represented by multi-value. 

50 14. An image processing method comprising the steps 
of: 

inputting image data; 

predicting pixel values indicated by the image 
data; 

encoding difference values between the pixel 
values indicated by the image data and predic- 
tion values predicted by said prediction step; 



division means for dividing an image into a plu- 
rality of block pixels; 

generation means for generating parameters 
indicating prediction values of the block pixels; 
conversion means for converting the values of 
the block pixels while controlling differences 
between the prediction values and the values 
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and 

controlling the difference values. 

15. An image processing apparatus comprising: 

s 

division means for dividing an image into block 
pixels to be encoded; 

generation means for generating parameters 
defining a plane based on values of first block 
pixels and values indicating positions of the "> 
block pixels to be encoded; and 
plane encoding means for encoding difference 
values between the value of the block pixels in- 
dicated by the plane defined by the parameters 
and the values of the encoded block pixels. 15 

16. An image processing apparatus according to Claim 
15, wherein said plane is a three-dimensional 
plane. 

20 

17. An image processing apparatus according to Claim 
15, wherein said plane is a proximate plane based 
on the values of the encoded pixels. 

18. An image processing apparatus according to Claim 25 
15, wherein the parameters are coefficients of a 
function representing the plane. 

19. An image processing apparatus according to Claim 

17, wherein said plane is determined based on sta- so 
tistics derived by calculating the values of the en- 
coded block pixels. 

20. An image processing apparatus according to Claim 

15, further comprising prediction encoding means 35 
different from said plane encoding means, wherein 
said plane encoding means and said prediction 
means are selectively used. 

21. An image processing method comprising the steps 40 
of: 

dividing an image into block pixels to be encod- 
ed; 

generating parameters defining a plane based 45 
on values of first block pixels and values indi- 
cating positions of the block pixels to be encod- 
ed; and 

encoding difference values between the values 
of the block pixels indicated by the plane de- so 
fined by the parameters and the values of the 
encoded block pixels. 
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